1. Field of Invention
This invention relates to man-machine interfaces for wireless communication devices, and more particularly, to man-machine interfaces constructed from markup languages.
2. Background of the Invention
Wireless communication devices are becoming an increasingly prevalent for personal communication needs. These devices include, for example, cellular telephones, alphanumeric pagers, xe2x80x9cpalmtopxe2x80x9d computers and personal information managers (PIMS), and other small, primarily handheld communication and computing devices. Wireless communication devices have matured considerably in their features, and now support not only basic point-to-point communication functions like telephone calling, but more advanced communications functions, such as electronic mail, facsimile receipt and transmission, Internet access and browsing of the World Wide Web, and the like.
Generally, wireless communication devices have software that manage various handset functions and the telecommunications connection to the base station. The software that manages all the telephony functions is typically referred to as the telephone stack, and the software that manages the screen display and processes user inputs of key presses is referred to as the Man-Machine-Interface or xe2x80x9cMMI.xe2x80x9d The MMI is the topmost, and most visible layer of the wireless communication device""s software.
Because wireless communication devices have generally reached a very desirable and small form factor, the primary determinant of a successful device will likely be in its feature set and its ease of use. Thus, the ability to quickly design, test, and deliver wireless communication devices that are both easy to use, and have a rich set of featuresxe2x80x94attributes that are often opposed to one anotherxe2x80x94will be essential to successful product performance.
However, wireless communication devices present a variety of more challenging design and implementation issues that do not arise with larger processor-based systems, such as notebook and desktop computers, which may also have similar telecommunication features. These design challenges include the design of the user interface, the customization of the devices for particular service operators, the integration of Internet and World Wide Web access with other communication functionality, and the software development process.
User Interface Design Constraints
Unlike desktop and notebook computers, wireless communication devices have a form factor which requires a very small screen display size. Desktop computers typically have displays with at least 14xe2x80x3 screen size, and resolution typically between 640xc3x97480 and 1024xc3x97768 pixels. In contrast, wireless communication devices typically have a screen size between 25xc3x9725 mm and 80xc3x97120 mm, and resolutions between 90xc3x9760 to 120xc3x97120 pixels, or about 3-8% of the size of the desktop or notebook screen. As a direct result, the user interface design of the wireless communication device must provide access to essentially the same features as desktop computers, such as electronic mail, facsimiles, and Web browsing, yet with only a fraction of the screen area for displaying text, images, icons, and the like. This problem of constructing the user interface to provide these features is particularly significant when handling Web based content, since conventional Web content, such as forms, assume the larger screen size of conventional desktop computers. Displaying such forms on the small screen of a wireless communication device results in jumbled and difficult to use content.
Another user interface limitation of wireless communication devices is the severely restricted set of inputs available to the user. Conventional desktop or notebook computers have cursor based pointing devices, such as computer mouse, trackballs, joysticks, and the like, and full keyboard. This enables navigation of Web content by clicking and dragging of scroll bars, clicking of hypertext links, and keyboard tabbing between fields of forms, such as HTML forms. Wireless communication devices have a very limited number of inputs, typically up and down keys, and one to three softkeys.
Accordingly, it is desirable to provide a software architecture for the MMI of a wireless communication device that enables the customization and use of user interface with Web content accounting for the limited screen resolution and input functionality of the wireless communication device.
Integration of Internet/Web Functional with Telephony
With the advent of the Internet and the World Wide Web, the highest performance wireless communication devices provide complete Internet access and the ability to directly browse the World Wide Web. Current devices provide Internet and World Wide Web access through a strictly modal interface, in which the user must select between using the wireless communication device in a browser mode, or in its native telecommunications mode for making telephone calls, accessing a stored telephone book, sending facsimiles, and the like. In the xe2x80x9cbrowser modexe2x80x9d the user cannot dial a telephone number to make a telephone call; likewise in the telephony mode, the user cannot access a Web site. Thus, the user is unable to operate the wireless communication device in a seamless fashion that allows Web content to be downloaded and manipulated in context of the telephone functions, such as embedding an item of Web content that is obtained while browsing into the user""s telephone book, or into an email message.
Accordingly, it is desirable to provide an MMI in which Internet and World Wide Web access features are seamlessly integrated with the telephony and other controls of the wireless communication device so that user can access any feature of the wireless communication device at any time.
Software Engineering of the MMI
Typically, an MMI is implemented as a module in a larger piece of code that manages the telephone control functions. The MMI is coded in the same computer language as the rest of the telephone control software. This makes the MMI difficult to modify without using the same programming skills and tools used to create the entire telephone control software. In other words, changing anything in the MMI requires the services of a skilled programmer familiar with the underlying telephony programming details and computer language. In addition, since the MMI is an integral part of the code for the telephone control software, implementing new changes in the MMI means compiling a new image of all the telephone control software, and testing the result to ensure that the new MMI features are compatible with all other code modules. In short, problems introduced by modifying the MMI software can potentially cause the handset to malfunction, disrupting service on the network to other users. Depending on the extent of the modifications, the change of any portion of the telephone control software can result in bugs, and/or the need for new type approval of the entire wireless communication device. Thus, it is desirable to provide a software architecture which separates the design and implementation of the MMI functionality from the implementation of the telephone control software, allowing the manufacturer to quickly and safely customize the MMI design to suit the needs of a particular customer
Customizing of the MMI for Service Operators: xe2x80x9cBrandingxe2x80x9d
In the wireless communication device industry, the services operators, such as cellular service providers, are interested in attracting and retaining their customers by aggressively branding their wireless communication device products, and offering new telephony features and network services to the user. Important among these are services that add value to the user, such as voice mail, electronic messaging, Internet access, and the like as mentioned above. xe2x80x9cBrandingxe2x80x9d is the embedding of insignia, logos, or other indicia into the MMI of the wireless communication device and its features that identifies it to the consumer as originating from the service operator.
The manufacturers of the wireless communication device, who typically provide only the basic hardware components, must therefore provide a way for the service operator to integrate these features and services into the wireless communication device by software programming, and provide a mechanism for branding the features. A key problem is that these services are necessarily different in their functionality and requirements, and the task of providing users with a current array of services and features is a difficult one.
Wireless communication device manufacturers have traditionally attacked this problem by making a special version of the wireless communication device control software for each service operator selling that wireless communication device in conjunction with its own communication services. Each specific version of the wireless communication device contains the device manufacturer""s branding, the operator""s branding, and support for whatever features and services the service operator supports. Each of these versions becomes a different piece of software to be tested, maintained, and modified as new features or services are provided to the consumer. This significantly increases the software development expense and maintenance issues. Further, unless the wireless communication device manufacturer provides the service operator with the source code of the MMI and telephone control software, it requires the wireless communication device manufacturer to be directly involved in the branding and MMI design requirements of the service operator. Thus, it desirable to provide a software architecture for an MMI that allows the wireless communication device manufacturer to provide a single body of telephone control software to each service operator, and allows each service operator to independently, and without the assistance of the wireless communication device manufacturer, design, implement, and brand the MMI for the wireless communication device.
The present invention overcomes the various limitations of conventional wireless communication devices by providing a wireless communication device with an MMI that is based on a markup language. A markup language is a computer programming language that allows the content of a page or a screen display to be defined by the inclusion of predefined symbols in the content itself indicating the logical components of the content, instructions for the layout of the content on the page or screen, or other data which can be interpreted by some automatic system responsible for displaying, manipulating or modifying the content.
In one aspect the present invention provides a wireless communication device including a user interface defined in a markup language. To effect this, the present invention includes a markup language browser that it uses to provide both telephony control of the wireless communication device, in response to user selection of telephony functions in the user interface, and Internet access via the HyperText Transport Protocol (HTTP), in response to user selection of data items associated with content located on the Internet.
In one embodiment, the telecommunication control and other functions of the wireless communication device are defined in various user interface pages written in a markup language. Each control function is associated with, or activated by a Uniform Resource Locator (URL). A URL is a data item specifying a protocol for obtaining a data item, and which data item should be fetched or manipulated. The user interface pages are stored in a local memory of the wireless communication device, and fetched by the browser, which decodes them and displays the appropriate user interface elements. The browser can also modelessly fetch markup language pages or other content that is stored remotely, by accessing such pages via a telecommunications network such as the World Wide Web, and likewise decode and display these remotely accessed pages. When a user interface page is displayed, user selection of a control function passes a URL or command data to the browser. The browser effects a telecommunication function in response the received URL or command.
The browser preferably includes a number of protocol handlers, including a telephony protocol handler, a local file protocol handler, and an remote file protocol handler, and a number of content handlers, including a markup language handler. The telephony protocol handler decodes URLs for telephony control features such as telephone dialing and answering, and activates underlying functions of telephony control software controlling the hardware of the wireless communication device. Any content of the URL that is needed to display the telephony controls is provided to the markup language content handler, which parses the content and displays it on a screen display. The markup language content handler is generally responsible for displaying any fetched markup language pages, including all user interface pages, and for receiving user inputs to these pages via forms and other input means.
The markup language handler generally receives content from two sources, the local file protocol handler and the remote file protocol handler. The remote file protocol handler decodes URLs for accessing content on the World Wide Web, and provides the fetched content, such as a Web page, form, applet, or the like to the markup language content handler for outputting the content to the screen display of the wireless communication device. One suitable remote file protocol handler implements HTTP. The local file protocol handler decodes URLs for accessing local user interface files and provides such content to the markup language content handler. In a preferred embodiment of the MMI, the user interface is defined in HyperText Markup Language, or xe2x80x9cHTML,xe2x80x9d and the browser includes a HTML content handler that displays both Web content and user interface featured defined in HTML.
The use of a markup language to define the MMI of a wireless communication device provides numerous advantages over conventional MMI software architectures. First, the use of a markup language allows for complete and seamless integration of Internet and World Wide Web access into the telephony and other features of the wireless communication device. Since the MMI uses a markup language such as HTML to display all the functional screens, the World Wide Web content (which is also written in HTML) has the same appearance as other features of the wireless communication device. More particularly, the pages of the MMI are accessed using URLs, just as Web content is similarly accessed. When displaying a functional page the wireless communication device accesses a local URL; when displaying Web content, the wireless communication device automatically initiates a connection with a Web server to obtain the Web content. The markup language based MMI thus allows for a modeless user interface that enables the user to access the Internet and the World Wide Web at any time, without having to switch the wireless communication device between telephony and xe2x80x9cbrowserxe2x80x9d modes, as in conventional devices.
As a further benefit of the markup language based MMI, Web content such as Web pages, forms, and the like, from the World Wide Web can be accessed and incorporated directly into telephony, messaging, and other non-Internet based features of the wireless communication device. For example, in a preferred embodiment, a wireless communication device has a telephone book of stored telephone numbers and names. Conventionally, the user would have to manually key these entries in using the keypad of the wireless communication device. In a wireless communication device using an MMI in accordance with the present invention, the user could add an entry to the telephone book simply by accessing a telephone directory on the World Wide Web, which can contain HTML that allows the user to easily store the information directly into the telephone book.
The use of a markup language also reduces the complexity of the software engineering process for creating the MMI for a particular wireless communication device. First, since the MMI of the present invention is based on a markup language, only a very limited amount of programming skill is needed to design a fully featured user interface, unlike a conventional MMI which requires a programmer skilled in C or other low level language programming. Editing and modifying the user interface requires only simple markup language and image editing tools, not a complete application programming environment. Second, using the markup language based MMI of the present invention enables any of the features the MMI to be modified, without having to re-compile the entire telephone control software, and re-test and certify the entire package. Because the MMI is separate from the underlying telephone control and air interface stack, only user interface pages that are individually changed or added need to be tested. This reduces the time to market, and increases the ease of designing, maintaining, and modifying the MMI as new features and services become available. Reduction of the time to create and test changes in the user interfaces also means that more different versions can be prototyped in less time than with a conventional MMI, thereby facilitating design exploration for the best user interface design for a given set of user requirements and features.
The ease with which the user interface of a MMI can be created and modified, and the reduction of time to market further enables the service operator to quickly generate wireless communication devices targeted at specific customer segments, without requiring the device manufacturer to create specific product software images for each and every target customer segment. For example, the service operator may use the same wireless communication device hardware and telephony control software with different user interfaces designed for executives, teen-agers and seniors, each of whom may have different needs and abilities to use the features of the wireless communication device.
For example, using a markup language to define the pages of the user interface allows any of the following items to be changed on any page: title bar presence and text; all informational text; option list text; links to all subsequent screens; soft key assignments; permanent scrolling banner messages; banner advertising; and help text.
The use of markup language based MMI also provides advantages in the branding of the wireless communication device for different service operators. Since the user interface is defined in markup language pages, service operator-specific logos, artwork, and text can be easily added and changed by individual service operators. Thus, the wireless communication device can provide the same wireless communication device hardware and telephone control software to a number of service operators, each of whom can quickly and easily brand the wireless communication device with their own distinctive user interfaces, without requiring the wireless communication device manufacturer to implement, test, and ship different user interfaces to each operator, as is conventional.
In providing a wireless communication device with a markup language based MMI, the present invention enhances the standard HTML with a number of extensions that make it particularly useful for working with wireless communication devices. Standard HTML assumes the presence of a conventional computer with keyboard, pointing device, and full size display screen, features which are not present in most wireless communication devices. The most notable deficiencies of HTML include:
Form elements (e.g., checkboxes, radio buttons) are awkward to navigate without a mouse.
Forms as they exist in content today tend to be too large for the user to maintain some context as she is filling them in on a small screen. If the form is divided into n forms, then the user""s input is sent between the client and the server and back to the client n-1 times, wasting bandwidth. In addition, with a series of smaller forms, terminating the transaction could be tortuous as the user hits the back key for each form in the series.
Hyperlinks are awkward to follow without a mouse to select them and a separate scrollbar for scrolling the content of a page. On a device with only an Up key and a Down key to both select which hyperlink to follow and to scroll the display, fixed assignment of either scrolling or selecting to the Up and Down keys is insufficient to provide the needed navigational abilities.
As a user interface definition language, HTML lacks a number of key features:
The ability to specify actions for the soft function keys, or indeed for any key on the device.
The ability to define a pop-up menu of choices.
The ability to display or alter the data one would like to store on the device, such as names and phone numbers.
The ability to design a screen as a template without writing C code to fill in the blanks.
The ability to allow content arriving over the air to extend or customize the interface the device presents to the user.
The present invention provides extensions to the HTML language which facilitate the design of multi-part forms, the use of a limited number of keys to both navigate Web pages and select hypertext links, define actions for any key (keypad or softkey) of the wireless communication device using URLs, create menus of options for softkeys, and conditional inclusion of text, formatting, and user interface gadgets.
More particularly, the present invention provides a xe2x80x9ckeyxe2x80x9d tag that allows the assignment of specific functions or actions to any key of a key-pad, including binding a menu to a key. A xe2x80x9ckeymenuxe2x80x9d tag allows specification of the menu items to be included in a menu bound to a key. A xe2x80x9ctemplatexe2x80x9d tag and an xe2x80x9cincludexe2x80x9d tag allow for the substitution or insertion of external HTML or other data directly into the HTML of a page. A xe2x80x9chelpxe2x80x9d tag allows for the definition of help strings that are automatically scrolled across the title bar of page after a set time period. A conditional tag allows for the testing of expressions to conditionally display HTML data within a page, for example based on variables or configuration settings of the device. A xe2x80x9cnextxe2x80x9d method for forms allows for maintaining state of a multi-part form without having to repeatedly transmit hidden data between a client and server to maintain the state. Improved navigational methods allow for the Up and Down keys of a wireless communication device to control both scrolling of a page, and selection of user interface gadgets and hyperlinks, in the absence of separate Tab and Enter keys and scroll bars.